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We investigate the potential for enhancing search sensitivity for signals having charm quarks in 
the final state, using the sizable bottom-mistagging rate for charm quarks at the LHC. Provided 
that the relevant background processes contain light quarks instead of charm quarks, the application 
of ^-tagging on charm quark-initiated jets enables us to reject more background events than signal 
ones due to the relatively small mistagging rate for light quarks. The basic idea is tested with two 
rare top decay processes: i) t —>■ c/i —>■ ebb and ii) t —>■ bH^ —>■ bbc where h and denote the 
Standard Model-like higgs boson and a charged higgs boson, respectively. The major background 
source is a hadronic top quark decay such as t ^ bW'^ —>■ bsc. We test our method with Monte 
Carlo simulation at the LHC 14TeV, and find that the signal-over-background ratio can be increased 
by a factor of 0(6 — 7) with a suitably designed (heavy) flavor tagging algorithm and scheme. 
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I. INTRODUCTION 

The discovery of the Higgs particle at the Large 
Hadron Collider (LHC) [1, 2] reaffirms that the Standard 
Model (SM) is a successful description of fundamental 
particles and their interactions in nature. Nevertheless, 
the detailed mechanism of protecting its mass scale from 
large quantum corrections is still unexplained by the SM, 
and new physics beyond the Standard Model (BSM) is 
anticipated to address this puzzle. Since the corrections 
are dominantly contributed by the top quark, the top 
quark sector has been regarded as a promising host to 
accommodate and reveal new physics signatures. Fur¬ 
thermore, the LHC, dubbed a “top factory”, is capable 
of copiously producing top quarks in pairs via the strong 
interaction, and it can therefore be taken as a great venue 
to discover new physics phenomena using top quarks. 

We emphasize that although many physical properties 
of the top quark have been measured with great preci¬ 
sion since its discovery, its decays are relatively poorly- 
measured; typical errors in the top quark decays are of 
0(10%) mostly coming from systematics in the measure¬ 
ment of the t-channel single top quark cross section ']. 
Hence, any new physics effects emerging in the top quark 
decay channels are, in principle, less constrained by cur¬ 
rent experimental data. 

One of the rare top decay examples to be considered 
here is t —>■ c/i via a flavor changing neutral current 
(FCNC) [5-8] followed by the doininant decay mode of 
the higgs of 125 GeV, i.e., h —>■ bb. In principle, noth¬ 
ing precludes the top quark from decaying in this man¬ 
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ner. Nevertheless, the SM prediction on the branch¬ 
ing ratio (Hr) of this process is extremely small due to 
the famous Glashow-Iliopoulos-Maiani mechanism and 
second-third generation mixing suppression, which re¬ 
sults in Br(t —>• ch)sM ~ 10 “^^ — 10 “^® [•■-’- 7 ]- There¬ 
fore, a significant excess from such a small SM expecta¬ 
tion could be a convincing sign of the existence of new 
physics. In fact, once new physics is introduced, the 
aforementioned suppressions can be relaxed, and thus 
fairly larger branching fractions can be anticipated, e.g., 
Br(t —)• ch)BSM ~ 10 “^ — 10 “® depending on the details 
of the BSM models of interest [ 7 ], which is comparable 
with the recent experimental bound reported by the CMS 
collaboration [9]. 

Another exciting scenario to be considered here is 
t —>■ where the charged higgs sequentially decays 

into a charm quark and an anti-bottom quark unlike the 
typical decay mode of —>■ cs. A sizable branching 

fraction of H~^ -T cb arises in a few models with two or 
more higgs doublets: for example, multi-higgs doublet 
models (MHDM) [10], flipped two-higgs doublet models 
(2HDM) [11-LI] with “natural flavor conservation”, and 
Aligned-2HDM [14]. Depending on the model details, 
Br(7J+ —)• cb) could be as large as ^ 80% [15]. Although 
existing experimental searches of t —> bH'^ —> bsc done 
by the CDF [16] and ATLAS [17] collaborations could 
be applied to the decay of —>■ cb [12], an enhanced 

branching ratio motivates more dedicated searches to dis¬ 
cover a new phenomenon or directly constrain the param¬ 
eter space in the relevant physics models. 

For the purpose of concreteness we focus on the collider 
signatures in the context of pair-produced top quarks, 
and assume that one of the top quarks decays into two 
bottom and one charm quarks via the decay sequences 
described above while the other follows the regular lep- 
tonic decay cascade. Provided with the visible final state 
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defined by the signal processes, obviously, the dominant 
SM background is semi-leptonic top quark pair produc¬ 
tion. Since there exist three bottom quarks for the signal 
process vs. two bottom quarks for the background one, 
the requirement of three bottom-tagged jets can substan¬ 
tially reduce background events. It is noteworthy that 
this event selection enables us to have the hadronic top 
quark decaying into bsc (i.e., t —>• bW^ —>■ bsc) as a 
main background source because the 6-mistagging rate 
for charm quarks is rather sizable. We henceforth take it 
as the major background unless specified otherwise. 

We point out that, remarkably enough, the high mis- 
tagging rate for charm quarks can be useful for a further 
improvement in the relevant signal-over-background ratio 
{S/B). More specifically, if one demands an additional 
bottom-tagged jet, then signal events can be selected by 
tagging the remaining charm quark as a bottom quark, 
whereas background events can be selected by tagging 
the remaining strange quark as a bottom quark for which 
the corresponding mis-tagging rate is typically far smaller 
than that for charm quarks. Therefore, we expect that 
the relevant signal sensitivity gets increased so that it 
is possible to probe smaller branching fractions of signal 
processes.^ Of course, a non-negligible reduction in the 
signal acceptance due to the additional &-jet requirement 
could be an issue. Given an immense production cross 
section of top pairs and a large expected integrated lu¬ 
minosity, for example, C = 300fb“^ at the 14TeV LHC, 
adequate statistics can be nevertheless achieved in these 
search channels. 


II. EXPECTED ENHANCEMENT AND 
POTENTIAL ISSUES 

To develop intuition on the basic idea described thus 
far, we provide a rough estimation of the expected en¬ 
hancement by parametrizing pertinent efficiencies. As 
mentioned before, a way to enhance the S/B (before any 
posterior analysis using kinematic variables) is to require 
one more 6-tagged jet in the final state, utilizing the siz¬ 
able mistagging rate of charm-induced jets. For more 
systematic comparison, we begin with (would-be) con¬ 
ventional selection scheme (denoted by 3b), that is, three 
bottom jets, one regular jet, and a W gauge boson. Since 
the W is irrelevant to the later discussion, we drop it for 
convenience. We first define some of the efficiencies with 
respect to the identification of bottom-initiated jets; bb 
as fo-tagging efficiency of b quark, be as &-mistagging ef¬ 
ficiency of c quark, and bg as &-mistagging efficiency of 
s quark (light quarks). With this set of definitions and 
S being the number of signal events before the tagging 


^ In general, the basic idea can be applied to the cases where the 
signal comes along with charm-induced jet(s) in the final state, 
whereas the counterparts in the background are light quark- 
induced jet(s). 


procedure, the expected number of signal events in the 
3b scheme (Ssh) is given by 

S3b = S{h^,(l-h,) + 3hX(l-bb)} , ( 1 ) 

where the first term represents the leading contribution 
while the second term represents the subleading contri¬ 
bution such as the case where c-induced jet is mistagged 
while one of the 6-induced jets is not tagged. When it 
comes to the major background, the leading contribu¬ 
tion comes from a hadronic W decaying into c and s as 
mentioned earlier. Therefore, the expected number of 
background events in the 3b scheme (Hsb) is 

H3b = Bh! |be(l - bg) + bg(l - be) + ^^(1 - bb)| ,(2) 


where B denotes the number of background events before 
the tagging procedure. We then have the S/B in the 3b 
scheme as 


^ bb(bb - 4bbbe + 3be) 

V ^ / 3b V ^ / bfcbc -I- bfebg -I- 2bcbg - 4b6bcbg ' 


Here we assume that the expected number of background 
events originating from t —>■ bdu is negligible because two 
light quarks are involved. 

On the other hand, if we modify the aforementioned 
selection scheme by requiring an additional 6-tagged jet 
instead of a regular jet (denoted by 4b), the expected 
numbers of signal and background events (iS' 4 b and i? 4 b, 
respectively) are expressed as 

54b = ShlK, (4) 

Bib = Bhlhchs , (5) 


from which the relevant S/B is simply given by 



4b 


\b) bg ’ 


( 6 ) 


where a small portion from t —> bdu is neglected again 
in computing i? 4 b. Defining the improvement of the 4b 
scheme with respect to the 3b scheme as I, we have 


(5/i?)4b 

(5/i?)3b 


bfcbc -|- bfjbg -I- 2bcbg — 4bbbcbg 
bg(b& - 4bbbc -I- 3bc) 


( 7 ) 


Since tagging efficiencies vary in the transverse mo¬ 
mentum of jets, it is interesting to investigate the depen¬ 
dence of I according to the Pt of 6-jets, which is explic¬ 
itly shown by red dots in FIG. I. As an example tagging 
scheme, the GSVM tagger of the GMS collaboration has 
been adopted, and relevant efficiencies are applied based 
upon the values reported in Ref. [18] wherein they have 
measured the data using tt events. To understand this 
behavior more intuitively, it is worthwhile to rewrite / 
using the leading contributions (i.e., the first terms) in 
eqs. (1) and (2): 


bc(l - bg) 
bg(l-bc)’ 


( 8 ) 
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One can easily see that the value of I is solely governed 
by be and bg. More specifically, this is an increasing func¬ 
tion as be (bg) increases (decreases) so that a large gap 
between be and bg is favored to attain a large improve¬ 
ment. In fact, it turns out that heavy flavor quarks are 
less tagged as &-jets while light quarks fake 6-jets more 
often in the high Pt region. The reason is that jets with 
a large Pt are typically collimated so that errors in par¬ 
ticle tracking, which is involved in the tagging algorithm, 
are likely to increase. As a consequence, less significant 
improvement is shown in the high Pt region. 

A couple of issues may arise in this strategy. Now 
that charm quark tagging techniques are being devel¬ 
oped [19], one could apply it to improve the S/B as an 
alternative option.^ To make a comparison of the idea in 
this paper with the data analyses involving the c-tagging 
technique, we again define relevant efficiencies with re¬ 
spect to the identification of charm-initiated jets; Cb as 
c-mistagging efficiency of 6 quark, Cc as c-tagging effi¬ 
ciency of c quark, and Cg as c-mistagging efficiency of s 
quark. Although there may exist some non-trivial corre¬ 
lation between heavy flavor tagging techniques and the 
possibility of 6-c mixing tagger [19, 23, 24], to be more 
conservative we ignore the events in which any of the 
visible entities involve a contradictory result between 6- 
tagging and c-tagging; for example, if a certain 6 is not 
only 6-tagged but c-tagged, the associated event is dis¬ 
carded. We basically require three 6-tagged jets together 
with one c-tagged jet. Denoting the tagging scheme ex¬ 
plained thus far as 3blc, we have the expected numbers 
of signal and background events (S'sbic and Bsbic, respec¬ 
tively) in this scheme as 

83 ^, 1 , = S {£!£',+ 3£^£,£'b} , (9) 

^sbic = B{£^£s£'^ + £^£,£i + 2£b£c£s£'b} , (10) 

where £i = bi(l — Ci) and £' = Ci(l — b^) {i = 6, c, s). 

The associated S/B is given by 

^ £bi£b£/ + 5£c£l) 

V ^ / 3bic V ^ / ^b£s£/ + £b£c£'s + ^£c£s£'b ’ ^ ^ 

and in turn, the associated improvement can be 
, ^ {S/BU3. ^ K ( £bi£b£/ + S£c£i) \ , , 

{S/BU bb 'v £b£s£/ + £b£c£'s + ‘i£c£s£'b 7 ' ^ ^ 

The blue squares in FIG. 1 demonstrate the Pt de¬ 
pendence of I'. For simplicity, the efficiencies associated 
with c-tagging are taken from the corresponding average 
values of the medium operating point in Ref. [19], that 
is, Cb = 0.125, Cc = 0.20, and Cg = 0.010. We see that I 


^ In fact, the ATLAS collaboration has already started to use the 
c-tagging technique in search for new physics, e.g.. Refs. [20, 21], 
while the usefulness of the c-tagging technique in rare top decays 
was mentioned, e.g.. Ref. [22]. 
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FIG. 1: Potential improvements I and I' of signal-over- 
background ratio as a function of Pt of jets in tt events. In 
this plot, the dependence on the rapidity of a jet is integrated 
out (1 17 1 < 2.4). The efficiencies associated with 6 -tagging 
are obtained from Ref. [18], whereas for I' the efficiencies 
associated with c-tagging are fixed to be the average values of 
the medium operating point in Ref. [19] for which (c 6 , Cc, Cs) ~ 
(0.125,0.20,0.010). 


is larger than I' in the entire range of Pt- This behavior 
can be viewed in a simpler way by considering the leading 
contributions (i.e., the first terms) in eqs. (9) and (10). 
We then have 


where I is the approximated expression in eq. (8). Since 
Cb is typically larger than Cg, I' is typically smaller than 
I. From this series of calculations, we find that the simul¬ 
taneous application of 6- and c-tagging techniques is not 
beneficial with respect to the signal-over-background ra¬ 
tio. Furthermore, the expected number of signal events 
itself (after applying the flavor tagging techniques) be¬ 
comes worse as is clear from the comparison between 
eqs. (4) and (9). Thus we employ only the 6-tagging 
technique for our data analysis. 

Another issue that one may argue is the possibility that 
other SM backgrounds come into play while an additional 
bottom-tagged jet is required. For the cases at hand, an 
immediate example is tthh for which one W gauge boson 
decays leptonically while the other is undetected. Cer¬ 
tainly, this issue depends strongly on the signal processes 
of interest, i.e., it may not be an issue for other signal 
models or collider signatures. We closely look at the im¬ 
pact of ttbb onto the relevant analysis, switching from the 
3b scheme to the 4b scheme in conjunction with Monte 
Carlo simulations later. 
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mH± (GeV) 

3b(xl0-=‘) 

4b(xl0"^) 

I 

tt 

- 

2.13 

0.0350 

- 

t ^ ch 

- 

24.7 

2.71 

6.68 


80 

20.4 

2.08 

6.21 


100 

20.6 

1.97 

5.82 

t bH+ 

120 

20.2 

2.04 

6.15 


140 

20.3 

2.13 

6.30 


160 

20.2 

2.11 

6.36 


TABLE I: Reduction rates for signal and background pro¬ 
cesses in 3b and 4b schemes, and the associated improve¬ 
ments /. The reduction rates for tt are computed with all 
decay modes included, whereas those for signal processes are 
computed only with the semi-leptonic decay mode. 

III. COLLIDER STUDY 

Equipped with the rough estimate discussed thus far, 
we test the feasibility of the basic idea in the context 
of the two aforementioned example models with Monte 
Carlo simulations of the 14TeV LHC. The parton-level 
events for the signal and the background are generated by 
MadGraph_aMCQNLD [25]. The output information is then 
streamed to Pythia 6.4 [2G] and DelphesS [27] in or¬ 
der. Jet formation is conducted by the anti-A:^ algorithm 
with a jet radius parameter R = 0.5. The 6-tagging effi¬ 
ciencies in a detector simulator for bottom, charm, and 
light quarks are tuned according to the performance of 
the CSVM algorithm reported by the CMS collabora¬ 
tion [18]. Furthermore, we apply cuts on the final state 
of both signal and background processes, closely follow¬ 
ing the selection scheme used in Ref. [28] for the semi- 
leptonic channel of top quark pairs. The key selection 
criteria for leptons and jets are highlighted below with 
small modifications: 

• for e and fi leptons, Pt > 10 GeV and \r]\ < 2.5, 

• for jets, Pt > 30 GeV and \r]\ < 4.9, 

• for 6-jets, Pt > 30 GeV and \r]\ < 2.4. 

We then require different 6-jet multiplicities for those 
selected events; for the 3b (4b) scheme, we exclusively 
demand 3 (4) 6-tagged jets and 1 (0) regular jets to¬ 
gether with an isolated lepton. Table I summarizes not 
only reduction rates of signal and background processes 
in both schemes but the resulting improvements. Six 
benchmark scenarios are examined here. To see the po¬ 
tential dependence on the choice of the charged higgs 
mass, we vary it from 80 GeV to 160 GeV at intervals 
of 20 GeV. The reduction rates for the ti sample are 
evaluated with hadronic and leptonic channels included, 
whereas those for the signal samples are evaluated only 
with semi-leptonic events. 

We observe that the relevant signal-over-background 
ratio can be improved by a factor of 0(6 — 7) for all sce¬ 
narios; no signihcant dependence is shown on the choice 
of the mass of the charged higgs. We point out that unlike 



FIG. 2: The distributions of Pt of bottom-tagged jets for 
background (black), t ^ ch (red), and t —>■ bH'^ (blue). For 
the charged higgs, a mass of 120 GeV is chosen as a represen¬ 
tative. 


the rough estimation demonstrated in FIG. 1, the overall 
improvement is reduced by a factor of ~ 2.5 from the 
maximum expected improvement. In more detail, signal 
efficiencies are reduced by ~ 10% from 3b to 4b schemes, 
which does not so much differ from the mis-tagging rate 
for charm quarks, whereas background efficiency is de¬ 
graded only by ~ 1.6% that is larger than the typical 
mis-tagging rate for light quarks. To understand this 
slight mismatch, one could suspect that a large fraction 
of 6-tagged jets come along with a large transverse mo¬ 
mentum (> 100 GeV) so that the overall improvement 
gets reduced, predicated upon the observation in FIG. 1. 
However, it turns out that only about a quarter of 6-jets 
belong to this hard Pt regime for both signal and back¬ 
ground events as clear from FIG. 2. In fact, this is not 
surprising, given charged or SM-like higgs masses them¬ 
selves and the mass gap between top quark and charged 
or SM-like higgs masses. Therefore, it does not make any 
substantial impact. We instead identify the effect from 
initial and final state radiations as a dominant cause of 
such a departure from the expectation in FIG. 1. More 
specihcally, initial or final state radiated gluons, which 
split into a heavy flavor quark pair, play a major role 
because those quarks are 6-tagged with high probability. 
In other words, once the contributions from ttbb or ttcc 
are switched on, the relevant efficiency drop for tt is not 
as large as expected, while such contributions appear as 
a subleading effect for signal processes. For the sake of 
validating this intuition, we perform another simulation 
of tt with initial and final state radiations completely 
turned off, and hnd that the efficiency drop between 3b 
and 4b schemes is restored to ^ 0.7%, which is close to 
the typical mis-tagging rate for light quarks. 
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IV. PROJECTIONS AND CONCLUSIONS 

As mentioned at the beginning, the LHC is capable 
of copiously producing top quark pairs, which makes 
the search channels discussed here become systematics- 
dominated.^ In more detail, the significance a is given 
by 


where k is a prefactor encoding overall systematics of 
backgrounds. Considering the typical k of 0(3%) for 
the tt channel (see, for example. Ref. [29]), we find that 
the second term in the denominator becomes larger than 
the first one once B is greater than ^ 1100. Note that 
the expected production cross section of top quark pairs 
at next-to-next-to-leading order including resummation 
of next-to-next-to-leading logarithm is 954 pb at the 
LHC 14 [30], and the branching ratio of tt —)■ bbcsiv is 
^ 15%. From these two numbers, we can easily see that 
the number of relevant background events B is much 
larger than ^ 1100 even with an integrated luminosity 
of 1 fb“^. Therefore, the relevant significance is propor¬ 
tional to S/B so that the improvements discussed in this 
letter can be directly translated into the associated sig¬ 
nal sensitivity, that is, for a given scenario, one can probe 
^6 — 7 times smaller branching fraction into the signal 
process of interest than expected in the 3b scheme. Ob¬ 
viously, posterior analyses with kinematic variables etc. 
can increase S/B further, which is beyond the scope of 
this paper. We instead leave such a research direction as 


future work. 

We again emphasize that the search strategy proposed 
here is not restricted to the benchmark scenarios em¬ 
ployed here, but straightforwardly extended to the sit¬ 
uations where the final state for the signal processes of 
interest contains charm quark-initiated jet(s) while the 
corresponding object(s) in backgrounds are light quark- 
initiated one(s). We also remark that different operating 
points may give rise to better improvements. Finally, we 
strongly encourage the ATLAS and CMS collaborations 
to adopt this idea as an alternative strategy in relevant 
new physics model searches. 
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